SINGULARITIES IN THE ENTROPY OF ASYMPTOTICALLY LARGE 

SIMPLE GRAPHS 



CHARLES RADIN AND LORENZO SADUN 



• Abstract. We prove that the asymptotic entropy of large simple graphs, as a function of 

04 ! fixed edge and triangle densities, is nondifFerentiable along a certain curve. 

(D 

^. 

1. Introduction 

Consider the set of simple graphs G with set V{G) of (labelled) vertices, edge set 
O ■ E{G) and triangle set T{G), where the cardinality = n. ('Simple' means the edges 

are undirected and there are no multiple edges or loops.) We will be concerned with the 
asymptotics of G"^ as n diverges, specifically in the relative number of graphs as a function 
of the cardinalities \E{G)\ and |T(G)|. 

Let Z^f be the number of graphs in G" such that the edge and triangle densities, e{g) 
and t{g), satisfy: 

cn- (1) e{G)=^-^j^e{e-a,e + a) and t{G) = ^-^j^ e {t - a,t + a). 

in : (2) (3) 

^ ■ Graphs g in U„>i(j" are known to have edge and triangle densities, {e{g),t{g)), dense in the 
O ■ compact subset R of the (e, t)-plane bounded by three curves, Ci : (e,e^''^), < e < 1, the 
line segment h : (e, 0), < e < 1/2, and a certain scalloped curve (e, h{e)), 1/2 < e < 1, 
lying above the curve (e, e(2e— 1), 1/2 < e < 1, and meeting it when e = Ck = k/ (fc+1), k > 
^1 1; see [PR] and references therein, and Figure 1. 

i We are interested in the relative number of graphs with given numbers of edges and 

triangles, asymptotically in the number of vertices. More precisely we will analyze the 
entropy density, the exponential rate of growth of the total entropy \ia{Z^f), as n grows, as 
follows. First consider 



(2) Sg'" = — and s{e,t) = lim lim s'^f. 

The limits defining the entropy density s{e,t) are proven to exist in |RSj . The objects of 
interest for us are the qualitative features of s{e,t) in the interior of R. Our main result is: 

Theorem 1.1. In the interior of its domain R the entropy density s{e,t) satisfies: 

(3) s(e,e^)-s(e,t) >c|t-e^| 



2010 Mathematics Subject Classification. 05C35, 05C30. 

This work was partially supported by NSF grants DMS-1208941 and DMS-1I01326. 

1 



CHARLES RADIN AND LORENZO SADUN 



triangle 
density t 




(0,0) 



(1/2,0) 
edge density e 



Figure 1. The phase space R, outhned in sohd hnes 

for some c = c(e) > 0. Therefore for fixed e, s(e, t) attains its maximum at t = but is not 
differentiahle there. For t < we have the stronger inequality 



(4) 

for some c = c(e) > 0. 



s(e, e^) - s{e,t) > c\t - 



So the graph of s{e, t) has its maxima, varying t for fixed e, on a sharp crease at the curve 
t = e^, < e < 1, and is not concave for t just below e^. 

We begin with a quick review of the formahsm of graph hmits; see |BCLSVt ILS11ILS21 ILS3j . 
and/or the comprehensive treatment in [Lovj . The main value of this formalism here is that 
one can use large deviations on graphs with independent edges [CV] to give an optimization 
formula for s{e,t) |RSj . 

2. Graphons 

Consider the set W of all symmetric, measurable functions 
(5) g ■.{x,y)e [0,1]^ ^g{x,y)e [0,1]. 

Think of each continuous set of vertices of a graph. For a graph G E G"' one 

associates 

(6) 



9^{x,y) 



1 if ( \nx] , \ny] ) is an edge of G 
otherwise, 



where ly] denotes the smallest integer greater than or equal to y. For g E W and simple 
graph H we define 



(7) 



t{H,g)= / TT g{xi,Xj)dxi---dxe 
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where ^ = \V{H)\, and note that for a graph G, t{H,g'^) is the density of graph homomor- 
phisms H ^ G: 

|hom(g,G)| 
\V{G)\\nH)\- 

We define an equivalence relation on W as follows: f ^ g ii and only if t{H^ f) = t{H, g) for 
every simple graph H. Elements of W are called "graphons", elements of the quotient space 
W are called "reduced graphons", and the class containing (7 G W is denoted g. The space 
W is compact in the metric topology with metric: 

(9) = 

where {Hj} is a countable set of simple graphs, one from each graph-equivalence class. 
Equivalent functions in W differ by a change of variables in the following sense. Let S 
be the space of measure preserving bijections a of [0,1], and for / in W and a G S, let 
fa{x, y) = f{a{x), <j{y))- Then f g ii and only ii g = for some cr G S. Note that if each 
vertex of a finite graph is split into the same number of 'twins', each connected to the same 
vertices, the result stays in the same equivalence class, so for a convergent sequence g^^ one 
may assume |V^(G'j)| — )• 00. 

The following was proven in [RSj . 



Theorem 2.1. ( |RSj ) For any possible pair (e, t), s(e, t) = max[— /((?)], where the maximum 
is over all graphons g with e{g) = e and t{g) = t, where 



e{g) = / g{x,y)dxdy, t{g) = / g{x,y)g{y, z)g{z,x) dxdydz 
and the rate function is 

(10) I{g) = I Ioig{x, y)) dxdy, where Io{u) = ]-[u ln(u) + (1 - m) ln(l - u)] . 

3. Proof of Theorem 11.11 

Proof. Fix a graphon g with edge density e. We can always write such a graphon as g = ge+Sg 
where ga is the constant function on [0, 1]^ with value a. We then compute 

J[0,l]2 i[0,l]3 

(11) + / 6g{x,y)6g{y,z)6g{z,x)dxdydz. 

J[0,l]3 

The first term on the right hand side is zero, since / 5g{x, y) dxdy = (5e = 0. If we think 

i[0,l]2 

of 5g as the integral kernel of the Hermitian trace class operator Tsg on -^^^([0, 1]), then using 



4 CHARLES RADIN AND LORENZO SADUN 

the inner product ( • , • ) and trace Tr we can rewrite the remaining terms as 

(12) <5t = 3e(0i,T40i)+Tr(T4), 

where = 1 is the constant function on [0, 1]. Note that the first term is non-negative. 



Using again the fact that / dg{x,y) dxdy = 0, 

J[0,112 

51 ^ J J/o(e + Sg{x,y)) - dg{x,y)I'Q{e) - /o(e)] dxdy 

= /' ^o(e + ^9{x^ y)) - S9{x yK{e) - 7o(e) ^^^^ 

j[o,i]2 mx,yy 

(13) > /_(e) / 6g{x,yydxdy, 

J [0,1] 2 

Io{e + x) -a;/^(e) - /o(e) 

///(e) 1 



where 



(14) f{e,x)^ 



and /-(e) = inf /(e, x) is a positive number less than or equal to 

Lemma 3.1. |Tr(T|^)| < {Tr{Tl^)fl'^, with equality if and only ifTsg is a rank 1 operator. 

Proof. Since Tsg is an Hcrmitian trace class operator it has pure discrete spectrum. If {/Xj} 
are the eigenvalues of Tsg, then 

(15) \Tr{Tl)\ -\Y.t^l\<Y. < max|/.,| ^/..^ < {J^t^lf' = [TriT^r'. 

i i i i 

If Tsg has rank one, then Tr{Tfg) = /i^ = ±[Tr(T|^)]^/^. If T^g has rank bigger than 1, then 

□ 



max(//j) is strictly smaller than ,^J^^J^- 



We next give an estimate for I{g) when t < e . If St < 0, then 

SI 



/-(e) 



(16) -St = -Tr{Tl) - 3e(0i, T.^O < -Tr(T4) < [TrlT^)]^'/^ < 
This implies that 

(17) SI > f4e){-St)'/\ 
Using < this also implies a linear estimate 

(18) SI > ^^\St\ 

e 

for St < 0. 

Finally, we estimate I{g) when t > e^. Since T^^^i) < rr(r5g), and since Tr{T^g) < 1, 
we have 

(19) St < Tr{Tl) + 3eTr{Tl) < {TriTsgff' + ^eTr{Tl) < (3e+l)Tr(T4) < 
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SO 

f4e)5t 



(20) 61 > 



3e + 1 ■ 

□ 



4. Other graph models 

We now generalize Theorem 11.11 to graph models where we keep track of the number of 
graph homomorphisms H ^ G for some fixed graph H, not necessarily triangles. We can 
compute the entropy of graphs with e{g'^) within a of e and t{H,g'^) within a of t, and 
define the entropies Sg'f and s{e,t) exactly as in equation The proof of Theorem 12.11 
carries over almost word-for-word to show the following. 

Theorem 4.1. For any possible pair {e,t), s{e,t) = ma.x[—I{g)], where the maximum is 
over all graphons g with e{g) = e and t{H,g) = t. 

Note that if H has k edges the constant graphon g^ satisfies t{H,ge) = . 

Theorem 4.2. For fixed < e < 1 the entropy density s{e,t) achieves its maximum at 
t = and is not dijferentiable with respect to t at that point. 

Proof. Following the proof of Theorem II. 1^ we write g = ge + 6g and expand both I{g) and 
t{H,g) in terms of 6g. The estimate (fT3|l still applies. The only difference is the expansion 
oit{H,g). 

Since t{H,g) is the integral of a polynomial expression in g, we can expand St as a poly- 
nomial in 6g. This must take the form 



dt = hi{x,y)6g{x,y) dxdy + / h2{w,x,y, z)6g{w, x)6g{y, z) dwdxdydz 



(21) +/ h3{x,y,z)6g{x,y)6g{y,z)dxdydz^ , 

^[o,ip 

where the non-negative functions hi{x,y), h2{w,x,y, z), h^{x,y,z), etc., are computed from 
the graphon from which we are perturbing. However, that graphon is a constant ge, so each 
function hi is also a constant. Thus there are non-negative constants ci, C2, . . ., such that 

5t = ci I 5g{x,y)dxdy + C2 5g{w, x)5g{y, z) dwdxdydz 

(22) + C3 / 6g{x,y)6g{y,z)dxdydz^ 

The first two terms integrate to zero, the third is positive semi-definite, and all subsequent 
terms scale as 5g^ or higher. 

For 5t < 0, we must have {—5t) of order I / 5g{x, yY dxdy J or smaller, which again 

Vi[o,i]2 / 

yields the analogue of equation (jlj), while for St > we simply note that, from ( IT3!) . 61 >0 
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which imphes s(e,e ) > s(e, t). Together these prove that s(e,t) cannot be differentiable in 
t att = e^. □ 

Note: There do exist some graphs H, such as "fc-stars" with k edges and one vertex 
on all of them, such that the lowest value of t for fixed e is on the 'Erdos-Renyi curve', 
t = e^, 0<e<l. For such graphs the analysis of what happens for (5t < is moot and 
s(e, t) may have a 1-sided derivative at {e,e'^). 

5. LEGENDRE TRANSFORM AND EXPONENTIAL RANDOM GRAPHS 

We return temporarily to the special case in which if is a triangle. Note that it has been 
fundamental to our analysis to use the optimization characterization of s(e, t) of Theorem 12 .11 
(Theorem 3.1 in [RSj ) . Treating this as an optimization with constraints, one might naturally 
introduce Lagrange multipliers Pi,P2 and consider the following optimization system, 

(23) max[-/(^) + Pie{g) + /32t(^)]; e{g) = e; t{g) = t, 

a 

namely maximize 

(24) ^/3i,/3.(^?) = -l{g) + Pie{g) + mg) 

for fixed (/3i, (32) and then adjust (/3i, /32) to achieve the desired values of e{g) and t{g). The 
'free energy density' 

(25) 7/'(/3i,/32) =max^;3,,^,(^) 

9 

is directly related to the normalization in exponential random graph models and basic infor- 
mation in such models is simply obtainable from it [Nl ICDt IRYt lARj . It can be considered 
the Legendre transform of s(e, t), but since the domain of s(e,t) is not convex the relation- 
ship between s(e, t) and ip{Pi,/32) must be more complicated than is common for Legendre 
transforms, in which convexity plays a significant role. In particular, although it has been 
proven ( [CDl IRY] ) that ipif^i, (32) has singularities as a function of (/3i,/32) (see Figure 2) it 
does not seem straightforward to use this to prove singularities in s{e,t), which therefore 
necessitated the different approach we have taken here. We will try to clarify the relation- 
ship between ip{(3i,(32) and s{e,t) through differences in the optimization characterizations 
of these quantities. 

As one crosses the curve in Figure 2 by increasing P2 at fixed the unique graphon 
maximizing \E'/3i,/32(5') jumps from lower to higher value of e{g), but still t{g) = e{g)^ [CDllRYj . 
We emphasize that whenever /32 > —1/2, one is on the Erdos-Renyi curve t = indicated in 
Figure 1 \CT>\ IRY] . This is significant in interpreting the singularities of s(e, t) and ip^Pi, [32)- 
The singularities or 'transition' characterized in Theorem [LT] and associated with crossing the 
Erdos-Renyi curve is presumably between graphs of different character but similar densities; 
we expect that graphons maximizing s(e, t), for t > e^, are related to those on the upper 
boundary of its domain R, while for t < they are related to those on the lower boundary 
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Figure 2. The curve of all singularities of ip{(3i,(32), for (32 > -1/2 



of R. (The latter are the subject of |RSj . ) On the other hand, the transition in Figure 2, 
associated with varying {(3i,(32), is between graphs of similar character (independent edges) 
but different densities, a phenomenon unrelated to the transition of Theorem 11.11 although 
still associated with the Erdos-Renyi curve, and which we understand as follows. 

Assume one optimizes ^E'/3i,/32(5') for fixed (/3i,/32), where (/9i,/32) is adjusted so that maxi- 
mizing graphons g satisfy e{g) = e and t{g) = t to match the desired values of (e, t) in which 
we are interested for s(e, t). It may happen that for special (/3i, (32) there are also optimizing 
g with other densities, {e{g),t{g)) ^ (e, t). This degeneracy is what is occurring precisely for 
the {(3i, (32) on the singularity curve of Figure 2. All such g clearly solve the maximization 
problem for s[e{g),t{g)]] they are appearing together when we fix {(3i, (32) because the value 
of ^^^,^2(5') happens to be the same for all these g, a phenomenon of no particular relevance 
to the original optimization problem of s{e,t). So in this sense degenerate solutions in the 
Lagrange multiplier method can be misleading; they point to a 'transition' which is foreign 
to the maximization problem for s{e,t). We next consider other features of the Lagrange 
multiplier method. 

One issue of importance to those who study exponential random graph models is that no 
matter how large (32 is, one cannot have maximizers g with t{g) > e{g)^ , though they certainly 
exist, as we see for instance from Figure 1. There are however less well known examples of 
'missing' values of (e,t) when using the Lagrange multipliers (/3i,/32), for t{g) < e{g)^, which 
have a different origin. 

5.1. The special case of e = 1/2. When e = 1/2, the n*'* derivative Iq^\x) is positive 
for n even and zero for n odd. This means that [/o(e + x) — Io{e)]/x'^ is a convex function 
of (since it is a power series in with positive coefficients). This allows us to find a 
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formula for 5g that simultaneously maximizes —Tr{Tgg) for fixed Tr{Tgg), minimizes the 
positive-definite quadratic term in 5t (to be zero) , and minimizes 51 for fixed Tr{Tl). This 
must therefore be a minimizer of the rate function. 

Lemma 5.1. Let Tsg be a rank-one operator: Tsgf = c{a, f) a where {a, a) = 1. Then 
St = c\ 

Proof. Assume c 7^ (or else the lemma is trivial). Since c{gi, a) {a, gi) — / Sg{x, y) — 0, 

i[0,l]2 

we must have ((71, a) — 0. This makes the quadratic term 3e((7i, Tg^gi) identically zero. Since 
Tsg is rank one with unique eigenvalue c, St — Tr{T^ 



□ 



a{x) 



Now we try to minimize / I[e + ca{x)a{y)\ dxdy. By convexity, this is minimized when 

i[0,l] 

[a{x)a{y)Y' is constant, which means that a{xY is constant. Since the integral of a is zero, 
we must have Q;(a;) = +1 on a set of measure 1/2 and —1 on a set of measure 1/2. Up to 
measure-preserving automorphism, we can assume that 

1 a;>l/2; 
-1 a;<l/2 

This means that the graphon that minimizes I{g) for fixed e = 1/2 and fixed t < is 

e + e X < 1/2 < y OT X > 1/2 > y 
e — € x,y < 1/2 or x,y > 1/2, 

where e — {e^ — tY^^, and that the associated reduced graphon is unique. The associated 
entropy is then 



(26) 



(27) 



A N -1 



since Iq{u) = Iq{1 — u) . 

Now consider the optimization using Lagrange multipliers. The Euler-Lagrange equations 
are: 



(28) 

where 

(29) 



h{x,y) 



[0,1] 



9{x,z)g{y,z) dz 



is the first variation of t{g) with respect to g{x, y). For our g — g, this becomes: 



/3i + 3/^2 



(30) 



^1 + 3/32 ( - + e2 



+ e 



In 



2 



+ e 



1 




2 


— e 


1 




2 


— e 



+ e 
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which are satisfied if and only if 

(31) fe = -ift = ^°»-^)-/a^+^), 1 

Notice that (3i and diverge as e J, (equivalently, as t f 1/8). Actually the situation 
is much worse. s(l/2,t) = —I{g) is a smooth function of t < 1/8. It is easy to check by 
differentiation of fl27|) that there are 0<ci<C2<l/8 such that s(l/2,t) is strictly concave 
on (0,ci) but strictly convex on (c2, 1/8]. Convexity implies that g is not a maximizer for 
^{Pii P2) for C2 < t < 1/8, but rather a local minimizer with respect to variation of t, and so 
there are no {Pi, P2) which can lead to the maximizers of s(l/2, t) for t just below 1/8. This 
phenomenon is simply due to inequality (jl]), and therefore occurs for all e, not just e = 1/2. 
In fact from the proof of Theorem 14.21 this phenomenon can be extended to subgraphs H 
other than triangles. [However, as noted above, for some H the Erdos-Renyi curve is actually 
the lower boundary of the domain of the entropy, in which case there are no 'missing' points 
(e, t) below it.] 

REFERENCES 

[AR] D. Aristoff and C. Radin, Emergent structures in large networks, J. AppL Probab. (to appear), 
arXiv:1110.1912 

[BCLSV] C. Borgs, J. Chayes, L. Lovasz, V.T. Sos and K. Vesztergombi, Convergent graph sequences I: 
subgraph frequencies, metric properties, and testing. Adv. Math. 219 (2008) 1801-1851. 

[CD] S. Chatterjee, and P. Diaconis, Estimating and understanding exponential random graph models, 
arXiv: 1102.2650v3. 

[CV] S. Chatterjee and S.R.S. Varadhan, The large deviation principle for the Erdos-Renyi random 

graph, Eur. J. Comb. 32 (2011) 1000-1017 
[Lov] L. Lovasz, Large networks and graph limits, American Mathematical Society, Providence, 2012. 
[LSI] L. Lovasz and B. Szegedy, Limits of dense graph sequences, J. Combin. Theory Ser. B 98 (2006) 

933-957. 

[LS2] L. Lovasz and B. Szegedy, Szemeredi's lemma for the analyst, GAFA 17 (2007) 252-270. 
[LS3] L. Lovasz and B. Szegedy, Finitely forcible graphons, J. Combin. Theory Ser. B 101 (2011) 
269-301. 

[N] M.E.J. Newman, Networks: an Introduction, Oxford University Press, 2010. 

[PR] O. Pikhurko and A. Razborov, Asymptotic structure of graphs with the minimum number of 

triangles, arXiv:1203.439f 
[RS] C. Radin and L. Sadun, a rXiv: 1301 .1256 1 

[RY] C. Radin and M. Yin, Phase transitions in exponential random graphs, Ann. Appl. Probab. (to 
appear), , arXiv:1108.0649l 

Charles Radin, Department of Mathematics, The University of Texas at Austin, Austin, 
TX 78712 

E-mail address: radin@math.utexas.edu 

Lorenzo Sadun, Department of Mathematics, The University of Texas at Austin, Austin, 
TX 78712 

E-mail address: sadunOmath . utexas . edu 



